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(54) Distributed internet protocol-based real-time multimedia streaming architecture 



(57) Multiple media push engines communicate 
with the multimedia client through a multi casting net- 
work that may incorporate multiple delivery paths. The 
streaming data representing media selections for deliv- 
ery are distributed across multiple media push engines 
using a non-hierarchial coding technique in which the 
data are represented as a set of substream compo- 
nents, capable of being reconstituted from fewer than all 
of the components of the original data stream. The 
higher the number of components used in reconstitu- 
tion, the higher the quality of service is provided by the 



reconstituted stream. Admission control to the group 
multicast session is administered in a distributed fash- 
ion, where an admission control unit opens the multicast 
stream, with all subsequent admission control decisions 
being made by the media push engines themselves. 
Substream component data are sent using Real-Time 
transport protocol while session management and the 
distributed admission control process are administered 
under the Real-Time Control Protocol. 
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Description 

Background and Summary of the Invention 

[0001] The present invention relates generally to net- s 
worked multimedia systems. More particularly, the 
invention relates to a media delivery system for deliver- 
ing media selections to one or more media clients over 
a multicasting network. 

[0002] With the explosive growth of the Internet, there . 10 
is a growing interest in using the Internet and other 
Internet protocol-based networks to deliver multimedia - ''. 
selections, such as video arid audio material. Interactive 
television, movies on demand, and other multimedia : 
push technologies are among the more promising app!i- ; is 
cations. v 
- [0003] The Internet is a connectionless network offer- 
ing best effort delivery service. Packets of data are 
routed as datagrams that carry the address of the 
intended recipient. A specific connection between the 20 
sender and the recipient is not required, because all 
host nodes on the network include the inherent capabil- 
ity to route datagrams from node to node until delivery is 
effected. This datagram packet delivery scheme is con- 
structed as a best effort delivery system in which the 25 
delivery of datagram packets is not guaranteed. Data- 
gram packets may be sent via different routes in the 
effort to increase the likelihood of delivery. Thus, if one 
node on the network is experiencing congestion, subse- 
quent datagrams may be alternately routed to avoid the 30 
congested node. This means that data datagram pack- 
ets do not inherently have a guaranteed arrival time. 
Even packets corresponding to a single message may 
be received out of order. This fact significantly affects 
how certain multimedia data are delivered. 35 
[0004] In many cases, multimedia data require real- 
time delivery. In the case of audio or video data, the 
data stream representing a particular media selection 
needs to be delivered in the proper time sequence, to 
allow the user to play back the audio or video selection 40 
"live" as it is being sent. Clearly, if the datagram packets 
are delivered out of order, due to taking different deliv- 
ery routes, then playback at the multimedia client (e.g., 
a user's interactive TV) will be jumbled. 
[0005] The Real-time Protocol (RTP) is a current de 45 
facto standard for delivering real-time content over the 
Internet (or other networks based on an IP protocol). 
The Real-time Protocol replaces the conventional trans- 
mission control protocol (TCP) with a framework that 
real-time applications can use directly for data trans- so 
port. Currently, tha RTP standard supports a first type of 
message;' namely one for carrying the media content 
data or streaming data. Typically, a separate r protocol,, 
the Realtime Control Protocol (RTCP) is used with - 
RTP to pass control messages for session manage- : 55 
ment, rate adaptation and the like. • ' ~ 
[0006] While the Real-time Protocol can be used -to 
deliver multimedia' sfreamirig data "over computer nst- " : 



works, the existing architecture has not prw robust 
enough to provide high quality presentation ; : best 
effort network services such as those providsc by the 
Internet. The present invention solves this problem by 
using a distributed media push architecture that is capa- 
ble of supplying streaming data redundantly from multi- 
ple sources and over multiple distribution paths. The 
media push engines have associated media storage 
units that store streaming data as non-hierarchial sets 
of substream components. The components are capa- 
ble of being reconstituted into a reconstructed stream 
from fewer than all of the components, such that the 
higher the number of components used in reconstitu- 
tion, the higher the quality of the reconstructed stream. 
[0007] Conventional systems use a hierarchial coding 
scheme that treats some components as more impor- 
tant than others. Thus, conventional systems typically 
need to expend considerable resources to guarantee 
that the more important components are always deliv- 
ered. In contrast, the media delivery system of the 
invention uses a non-hierarchial coding scheme, multi- 
ple description coding (MDC) that treats all components 
as equals. Thus, no special resources need to be allo- 
cated to ensure that a given set of substream compo- 
nents is delivered. Naturally, the higher number of 
components delivered, the higher the quality achieved; 
on the other hand, unlike with conventional hierarchial 
coding, loss of any given single packet does not appre- 
ciably degrade the signal quality. 
[0008] The distributed media delivery system also 
employs a distributed admission control system. The 
media client contacts a single admission control unit to 
request a given media selection, but thereafter the 
admission control decisions are handled in distributed 
fashion by the media push engines themselves. The 
admission control unit communicates the request to a 
plurality of media push engines distributed across the 
network and those push engines individually determine 
whether they can participate in the multicasting session. 
Thus, the individual media push engines each evaluate 
local traffic congestion to determine whether it is capa- 
ble of supplying the requested data stream. The admis- 
sion control unit: is thus not involved in directly 
determining which media push engines should be 
admitted to a multicast group session. The admission 
control unit simply assigns the multicast group session 
address and then allows the admission process to pro- 
ceed autonomously, in a distribLrted fashion. 
[0009] For a more complete understanding of the 
invention, its objects and advantages, reference may be 
had to the following specification and to the accompany- 
ing drawings. 

Brief Description of the Drawings 
[0010]. 

Figure 1 is a network datagram illustrating a pre- 
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f erred embodiment of the invention; 
Figure 2 is a detailed network datagram showing 
how two different data streams (X and Y) are dis- 
tributed across the network using non-hierarchial 
multiple description coding ; 5 
Figure 3 is a data flow diagram illustrating one 
embodiment of multiple descriptive coding (MDG); 
Figure 4 is a iayer datagram illustrating. the TCP/IP 
architecture and also showing how the RTP archi- 
tecture may be integrated into an IP-based system; 10 
Figure 5 is a detailed format datagram illustrating 
the packet format according to the Real-time Proto : 
col (RTP); ' 
Figure 6 is a network datagram illustrating the call- "■"■■<• 
admission and session management according td ; .:is 
the invention; ^ . 

Figure 7 is a detailed network datagram showing- . 
information flow between a media push engine and - 
a multimedia client over the multicasting: IP net- 
work; ■ ' . ■ - --.'SO 
Figure 8 is a network datagram illustrating the proc- 
ess of source component server redistribution; and 
Figure 9 is a protocol datagram showing how the 
Real-time Protocol (RTP) may be modified to 
increase its reliability. 25 

Detailed Description of the Preferred Embodiment 

[001 1 ] Referring to Figure 1 , an exemplary distributed 
networked multimedia system is illustrated at 1 0. A plu- 30 
rality of media push engines 12 are accessible through 
the multicasting network 14. The presently preferred 
embodiment is designed to work over a network 
employing the Internet protocol (IP); however, the prin- 
ciples of the invention may be readily extended to net- 35 
works using other protocols. The network 14 is also 
assessable by one or more multimedia clients 16, as 
illustrated. An admission control unit 18, accessible 
through network 14, performs certain admission control 
functions, primarily to initiate or open a multicast group 40 
session. The admission control unit includes a catalog 
services system 20. The catalog services system con- 
tains a database record indicating which multimedia . 
selections are available for delivery by the plurality of 
media push engines. Although involved in. openingira ->.4s 
multicast group session, the admission control process \: 
is actually performed in a distributed fashion as will be - 
more fully discussed below. 

[0012] The distributed media delivery system , 
responds to delivery requests from a multimedia client so 
by opening a multicast group session among the media 
client and those multimedia push engines that have the 
requested media selection available for delivery. Typi- 
cally multiple media push engines will participate in 
simultaneously delivering streaming data correspond- 55 
ing to the requested selection. The multimedia client is ■ 
a user host that performs the presentation function. It 
reconstructs a final stream from the various stream 



components delivered by the participating media push 
engines. Each media push engine has its own data stor- 
age for the stream component data and those data stor- 
age systems can be mediated by a suitable distributed 
file system that provides a mountable and transparent 
storage and retrieval function. 

[001 3] ; An important aspect of the distributed media 
delivery system is the manner in which streaming data 
are stored on the media push engines. Unlike traditional 
systems that store multimedia data in a hierarchial.fash- 
ion, the present invention uses a non-hierarchial coding 
scheme,. <ref erred to here as multiple description coding. 
(MDG). The multiple description coding splits the video 
and/or. audio, stream, into substreams called compo- 
nents; Each. component can tJhen be coded and trans- 
mitted over the network independently from all other 
components. The client software on a multimedia client 
16 can assemble a reconstructed stream from any sub- 
set of the components. Thus the reconstructed stream 
can be assembled from fewer than all of the compo- 
nents. The higher the number of components used in 
reconstruction, the higher the quality of the recon- 
structed stream. 

[0014] Using this non-hierarchial coding to deliver 
streaming data oyer the inherently unreliable network 
affords surprisingly robust media delivery, particularly 
when multiple push engines participate in the delivery. 
As will be more fully discussed, the media push engines 
control the multicast group session admission process 
themselves, in a distributed fashion, adding or subtract- 
ing media push engines to the group session as needed 
to maintain a high quality of services. Thus, when the 
multicasting network 14 exhibits low traffic congestion, 
only a few media push engines may be needed to sup- 
ply all of the components of the MDC-encoded stream. 
Even if some components are not delivered in a timely 
fashion, the multimedia client will nevertheless be able 
to reconstruct the stream for presentation (with some 
degree of degraded quality). If the network traffic con- 
gestion is high, the media push engines negotiate with 
one another to add additional media push engines. 
Because the admission control process is distributed, 
individual media push engines are able to assess their 
own local traffic congestion and will thus participate jn 
the group session, or not, depending on local traffic con- 
ditions. : v <:.:•..- 

[001 5] Figure 2 shows in grater detail how the multi- 
ple description coding works. In Figure 2, two multime- 
dia streams, designated X and Y are stored across a 
plurality of media push engines. These streams are bro- 
ken into substream components, designated by sub- 
scripts;- X J( X 2 . , Xp ^ Yj ,X 2 - - - - Note that,th£ 

substream components .stored across the plurality ot 
media ; push, engines are not .necessarily the same ,for 
.each engine. Thus media pusri. engine 12a stores com- 
ponents X 1t Xg and Y v . Similarly, media push, engine 
1 2b stores .components Xg, ^C 7 and Y 2 . . * 

[00|6] . The v multirriedia ciients^: reassemble the data 
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stream of interest by summing the proper substream 
components in the proper order. Thus multimedia client 
16a reconstructs stream X as illustrated, while multime- 
dia 16b constructs stream Y as illustrated. At the multi- 
media client it matters not that individual substream 5 
components arrive through different paths from different 
media push engines. 

[0017] The presently preferred multiple description 
coding scheme is constructed as illustrated in Figure 3 
The original multimedia data stream (e.g. video and/or 10 
audio data) is decomposed into multiple subsignals and 
then each subsignal is independently compressed. As " 
notes above; the decomposition is non-hierarchial, such- 
that an acceptable signal can be recovered from "any*- 
one subsignal, an incremental improvement can be--" /5 
realized by additional subsignals, and a perfect recon- 
struction is achieved when all subsignals are received ■ 
exactly. Moreover, it is preferable to maximize the over- 
all cr -repression gain so long as the above three criteria 
are :-v«. 20 
[00 ' : j One way to decompose the original signal is to 
construct each subsignal as a reduced resolution repre- 
sentation of the original signal. This may be done/ as 
illustrated, by submitting the original signal through a 
low pass filter followed by downsampling. The subsig- 25 
nals thus differ only in the sampling positions. If desired, 
such decomposition can be obtained by a filter bank 
that includes the pre-filter and its shifted versions. 
[0019] The pre-filter should be selected to suppress 
the aliasing components in the downsampled subsig- 30 
nals. This helps to reduce the bit rates required for cod- 
ing the subsignals and to enable an acceptable image 
recovery from a single subsignal. 
[0020] The pre-filter should further be selected so that 
it does not completely eliminate the high frequency 35 
components. Were the high frequency components 
entirely eliminated, there would be no way to recover 
those components in the original signal, even when all 
subsignals are present. Thus the filter should suppress, 
but not entirely eliminate, the high frequency compo- 40 
nents. 

[0021] Mathematically, reconstruction of the sub- 
stream components into a reconstructed stream 
involves inverting* the matrix equation relating the sam- 
ples in all the substreams and the samples in the origi- 45 
nal stream. Typically this may involve a large matrix 
equation, with inversion employing a large amount of 
computation and memory space. One way to address 
this computational burden is to use a block recursive 
reconstruction method. At each step in the recursive so 
process a block of 2x2 samples in the original stream 
is recovered based on up to four corresponding sam- 
ples in* iris received subcomponent streams. Of ccurse, 
other cornputationai' techniques may be employed to 
accomplish the same result: For more information on 55 
coding and encoding in a non-hierarchial fashion using 
multiple description coding," see "Robust Emage Coding 
and Transport in Wireiess Networks' Using aNon-hlier- 



archial Decomposition," Yao Wang and Doo-man 
Chung. Mobile Multimedia Communications; Goodman 
(Plenum Press) 

[0022] The MDC-coded substreams are delivered 
over the multicasting network as datagrams using the 
Real-Time Protocol (RTP) for content delivery and the 
Real-Time Protocol (RTCP) to implement flow control. 
These protocols are able to naturally coexist with the 
popular TCP/IP protocol used by many Internet applica- 
tions. Figure 4 presents a review of these protocols 
through an example of how five entities might communi: 
cate with one another. In Figure 5 entities 30-38 com- 
municate with each other using the layered architecture 
popularized by the Internet. It will, of course, be under- 
stood that Figure 4 is intended only to show how the 
presently preferred Real-Time Protocol (RTP) inte- 
grates in one possible architectural scheme. Although 
the Real-Time Protocol is presently preferred, this is not 
intended as a limitation of the invention in its broader 
: aspects. On the contrary, other message delivery proto- 
cols may be suitably used. 

[0023] In Figure 4 each of the communicating entities 
30-38 has been illustrated as a layered architecture, 
with the physical layer at the bottom and the application 
layer at the top. Entity 30 communicates with entity 32 
using the Ethernet Protocol at the physical level. Entity 
32 and entity 34 communicate at the physical level 
using the ATM Protocol. In a like fashion, entity 34 com- 
municates with entity 36 using the Ethernet Protocol, 
and entity 36 communicates with entity 38 using the 
PPP Protocol. Again, the physical layer communication 
protocols selected for illustration here are not intended 
as a limitation upon the invention as set forth in the 
appended claims. 

[0024] Above the physical layer is the Internet Proto- 
col (IP) layer. The IP Protocol isolates the physical or 
transport layer from the application layer. The IP Proto- 
col supports connectionless communication in wmdh 
packets of information are sent and received as cr-aica- 
grams. Note that in the illustrated example of Figure 4 
all communicating entities are using the IP Protocol. 
[0025] One layer above the IP Protocol two different 
higher-level protocols have been illustrated, the TCP 
Protocol and the UDP Protocol. Again, the illustrations 
are intended only as one example of a possible configu- 
ration. The UDP Protocol or User Datagram Protocol 
represents a simple transport protocol. It makes no 
attempt to preserve the sequence of messages it deliv- 
ers. The TCP Protocol or Transmission Control Protocol 
provides a higher level of reliability, yet insures that dat- 
agrams are delivered in the proper sequence. The TCP 
Protocol uses an acknowledgment system to insure that 
all datagrams are delivered in the proper sequence. The 
TCP Protocol includes a mechanism for retransmitting 
packets that have not been acknowledged. This 
acknowledgment/retransmit technique guarantees 
proper, packet delivery, . but it does not guarantee packet 
delivery.- in real-time. Thus TCP Protocol is . usually 
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unsuitable for delivering real-time data such as multime- 
dia video and/or audio data. 

[0026] The Real-Time Protocol (RTP) replaces the 
more sophisticated Transport Protocol of TCP with a 
simple framework that applications can use directly. 
Instead of implementing a missing data detection and 
retransmission mechanism-which can introduce trans- 
mission delays-the RTP Protocol simply ignores miss- 
ing data. The RTP Protocol is also not typically 
concerned with the sequence of packet delivery The 
protocol assumes that the application layer above it will 
correct any misordered data. The RTP Protocol is com- 
patible with a number of different encoding standards, 
such as MPEG, JPEG and H.26T. ' 
[0027] In the illustrated example of Figure 4, entities r 
30 and 38 are boih running RTP Protocol. Thus stream^ 
ing data could be supplied from entity 30 to entity 38 
through the network consisting of entities 30, 32, 34, 36 
and 38: 

[0028] The RTP Protocol is designed for multicast 
operation. Multicasting is a form of message broadcast- 
ing in which messages may be delivered to many differ- 
ent recipients in a designated set. Multicast addresses 
identify sets of interfaces, frequently including multiple 
interfaces belonging to different systems. When a mes- 
sage has a multicast destination address, the network 
strives to deliver it to all interfaces in the set. This func- 
tion lets a system generate a message once and have 
that message delivered to many different recipients. 
[0029] Aside from delivering the datagram packets to 
multiple recipients, a multicasting network will typically 
also support feedback from the message recipients. 
Typically all participants in the multicasting group ses- 
sion can receive these feedback messages. Such feed- 
back messages are commonly used for real-time traffic 
control, following a related Real-Time Control Protocol 
(RTCP). In some respects, RTCP is an optional exten- 
sion to RTP. RTCP packets are used by the presently 
preferred embodiment to send flow control and session 
management information among the entities participat- 
ing in the group multicast session. : 
[0030] Figure 5 illustrates the RTP packet format. 
Note that the packet includes a sequence number, and 
time stamp used in reassembling 'the packets in. the 
proper time sequence. 

[0031] Figures 6 and 7 show the details of how the 
multimedia client, admission control unit and media 
push engines communicate with one another. during a 
multicast group session. Specifically, Figure 6 shows 
the basic message flow and communication sequence 
of the preferred embodiment. Figure 7 gives a more 
detailed view of how the RTP and RTCP Protocols are 
used in routing the substrearh component ' datagrams 
from media push engine to multimedia client. 
[0032] Refen ing first to Figure 6, the multimedia client 
16 sends a unicast TCP Protocol message to the 
Admission Control Unit 18, requesting that a particular 
media selection commence delivery. The Admission 



Control Unit 18 consults its catalog services system 20 
to determine if the requested selection (i.e. requested 
stream) is present on the network. Assuming the stream 
is present, the Admission Control Unit transmits a 
5 Stream Open message to those Media Push Engines 
12 that have at least some substream components of 
the requested selection. This Open Stream request is 
sent - to all Media Push Engines. Those Media Push 
Engines that find themselves capable of serving the 
70 requested stream components jointly enter the multi- 
casting session management and flow control session 
between :Only those hosts serving and receiving that 
specif ic stream. Participating Media Push Engines and^ 
participating multimedia clients obtain the needed multi- ~ 
sis ca^t .address for such control multicast group session 
from.the Admission Control Llnii Thereafter,, the Admis- 
sion Control Unit effectively drops out of the session, 
allowing subsequent session management and flow 
control messages to be exchanged only among multi- 
. 20 . cast group members (that is, the multimedia client and 
all corresponding media push engines). This reduces 
the overhead on the Admission Control Unit 18. 
[0033] The Admission Control Unit generates a multi- 
castClass D address for use by the multicast session. 
25 This address may be selected from a pool of available 
multicast address entries. The Admission Control Unit is 
thus responsible for managing the allocation of multi- 
cast addresses. When a multicast session is ended the 
Admission Control Unit returns the multicast session 
30 address back to the pool of available addresses. 

[0034] Thus once the multicast group session is initi- 
ated, those media push engines able to supply sub- 
stream components will do so by sending unicast RTP 
session stream data to the multimedia client 16. 
35 Through the RTCP- Protocol these media push engines 
may communicate with one another to join or depart a 
multicast group session, as needed to maintain a high 
quality of service. 

[0035] Referring to Figure .7, the media push engine 
40 and multimedia client communicate through the network 
at two different levels. As illustrated by the dotted lines, 
a unicast RTP session transmits the multimedia stream- 
ing data to the multimedia client. Concurrently , as 
required, the media push engine and multimedia client 
45 send each other RTCP reports, specifically sender 
reports and receiver reports as well as any appropriate 
flow control commands and other session management 
commands < (e,g. Start Push, Pause, Continue). The 
RTCP control signals are shown by the bi-directional 
so solid line in Figure 7. - 

[0036] Essentially, after the Stream Open message is 
sent by : the Admission Control. Unit, each rn^j^push 
engine consults its associated media storage system jo. 
determine,- whether .- it : is .. capable .of." serving . the 
55 requested stream components. Jf so, Jhen the, .rpedia 
push enginajpins: the specif ied multicast group. "Other- 
wise dt> dqes nqt participate Jn;.iiie multicast session 
(ufiUesS'later req^e^e^^lO^^k "media, push engine. 
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has joined the multicast group, it participates in commu- 
nications using the RTCP Protocol whereby statistics of 
sent data and received data are exchanged among the 
members of the group. As noted above, the Admission 
Control Unit does not need to participate in these com- 5 
munications and hence it will remain dormant unless a 
request for another session is made or until the current 
session is requested to be terminated. 
[0037] Effectively, the system implements a distributed 
Admission Control System, where the participating 10 
members of the group collectively and distributivefy 
make the admission control decisions. One benefit of 
this distributed approach is that the invention can incor- ~ 
porate intelligent mechanisms to prevent network con- 
gestion and to improve quality of service, despite the 15 
fact that the multicasting network is a best effort network 
with no guaranteed real-time delivery. 
[0038] Best effort networks, particularly those lacking 
sophisticated traffic and user control policies, experi- 
ence frequent congestions. Such congestions can 20 
result in a loss of or substantial delay of real-time data. 
As previously discussed, real-time data that is delivered 
late is effectively treated as not delivered. The continu- 
ous influx of data into a congested node of the network 
tends to make the congestion even worse. The present 25 
invention uses RTCP sender reports and received 
reports on time-sensitive transmissions as an indication 
that a given node may need to scale back on the 
number of components it is transmitting when conges- 
tion is detected. 30 
[0039] Figure 4 illustrates how this may be accom- 
plished. The multimedia client 1 6 has requested the X 
real-time data stream, consisting of sub-stream compo- 
nents X 1t X 2 , X 3 and X 4 . Assume that Media Push 
Engine 12 in Figure 7 is experiencing local traffic con- 35 
gestion such that sub-stream components are arriving 
late at the multimedia client 16. The multimedia client's 
RTCP receiver report notifies the Media Push Engine 
12 (and all other media push engines participating in the 
group session) that some percentage of the component 40 
data from Media Push Engine 12. Media Push Engine 
1 2 analyzes these reports and stops sending a selected 
component, in this case the X 3 , thereby decreasing the 
amount of traffic flowing through its point of congestion. 
Thus, after adjustment, Media Push Engine 12 supplies 45 
only components X 1t X 2 and X4 to the multimedia client 
As the other media push engines participating in the 
group session receive the same sender and receiver 
reports, the loss of the X 3 component from Media Push 
Engine 12 may be compensated for if other push so 
engines are able to supply- this missing component. 
Otherwise, the quality of service will be slightly 
degraded as discussed above. "-• • • *- 
[0040] Figure 8 illustrates hew the data stream may be 
effectively redistributed by making local adjustments to ss 
the substream components being sent. In the illustrated 
example assume that there is local congestion some- 
where in the dafa path^hat-supplies substream compo^ • 
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nents from Media Push Engine 12b. The RTCP sender 
and receiver reports will thus indicate that some portion 
of the components previously sent to the multimedia cli- 
ent 16 by Media Push Engine 12b are lost or delayed 
due to local congestion. In the illustrated example the 
lost components happen also to be present in the stor- 
age system of Media Push Engine 12a. Media Push 
Engine 12a can either retransmit the lost component 
payloads to the multimedia client or simply adjust the 
set of components to be transmitted in future real-time 
data transactions. In the case where lost payloads are 
retransmitted by another media push engine, sufficient 
buffering should be supplied at the multimedia client to 
allow the missing components to be reassembled with, 
the previously delivered components before the stream 
is reconstructed and presented to the user. In the esse 
where the system merely alters the component set for 
future transmissions, such change constitutes a scala- 
ble server component redistribution mechanism. This 
mechanism promotes improved quality of service by 
improving the presentation of stream data to the multi- 
media client. 

[0041] Although the embodiment described above is 
generally suitable for most media delivery applications, 
there are some systems that cannot tolerate even a 
slight degradation in quality. Such systems include high 
quality broadcast video distribution. In these more 
demanding applications, the system of the previously 
described preferred embodiment can be modified to 
employ an additional reliability mechanism for the real- 
time component. In this case the Real-Time Protocol 
may be modified or augmented to allow retransmissions 
of lost real-time payloads. This "Reliable RTP" is illus- 
trated in Figure 9. The Media Push Engine communi- 
cates to the RTP stack using Real-Time Protocol. In this 
case assume that the first and third components are 
received but the second component is missing. There is 
an immediate negative acknowledgment (NACK) from 
the RTP stack, telling the Media Push Engine that the 
second payload. was not received. The Media Push 
Engine then retransmits the needed payload and the 
RTP stack places the needed payload into the correct 
location, within, the data buffer. The client application 
then reads the data from the data buffer. Any duplicate 
packets are dropped and any excessively delayed pack- 
ets may also be dropped. . 

[0042] From the foregoing it will be understood that 
the invention provides a media delivery system architec- 
ture that employs a distributed, networked technique for 
delivering streaming data over a best effort network. 
The architecture may be readily scaled, larger or 
smaller, as the server's complexity increases linearly 
with the number, of clients. The architecture is thus a 
fully distributed, tightly ccupled parallel architecture that 
is capable of providing simple and yet robust service. 
[0043] Through the use of multiple description coding 
(MDC) and multiple path transport, the invention can 
provide a high quality of, service without resorting to 
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delay-producing transmission retry techniques. Thus 
the invention will readily work with existing Real-Time 
Transport Protocol (RTP) for data transport and Real- 
Time Control Protocol (RTCP) for session manage- 
ment, rate adaptation and the like. When congestion is 5 
encountered the presentation can be downscaled with- 
out interruption, thanks to the multiple description cod- 
ing and the manner in which participants of a group -■ 
session can be added to or removed from the group. ».■• 
The flow control of streams is also controllable through }0 
these same mechanisms to prevent or reduce network 
congestion once it is detected. ■ * 

[0044] The invention is therefore ideally suited for ' - 
delivery of multimedia selections as well as Video arid '» 
audio streaming data. The invention will readily support - 75 
data streams of multiple bit "rates and it is capable of 
providing services at both constant bit rates arid varia- ■ - 
ble bit rates. 

[0045] While the invention has been described in its ; * 
presently preferred embodiments, it will be understood ~ 20 
that the invention is capable of certain modification iand* 
change without departing from the spirit of the invention 
as set forth in the appended claims. 



Claims 



A distributed media delivery system for delivering 
media selections to a media client (16) over a mul- 
ticasting network (14), characterized by: 



25 



30 



a plurality of media push engines (12) accessi- 
ble through said network (14), said push 
engines (12) each having associated media 
storage unit or storing streaming data repre- 
senting the media selections available for deliv- 35 
ery; 

said media storage units being configured to 
store said streaming data as a non-hierarchical 
set of substream components capable of being 
reconstituted into a reconstructed stream from 40 
fewer than all of said components, such that 
the higher the number of components used in 
reconstitution, the higher quality the recon- 
structed stream; and 

an admission control system (18, 20) accessi- 45 
ble through said network (14), said admission 
control system (18, 20) including a. catalog: for, 
storing the identity of the media selections^ 
available for delivery by each of said media 
push engines (12), - so 

said admission control system (18, 20) being 
operative, in response to a request for a given 
media selection from a media client (16), to . 
open a multicast group session among said 
media client (16) and at least a portion of said 55 
media push engines (12) having the given 
media selection available for delivery, - J ''• 
whereby said media push engines participating- . 



in said multicast group session each supply to 
said network those substream components 
corresponding to the given media selection, for 
delivery to and reconstitution by said media cli- 
ent. 

2. The media delivery system of claim 1 , character- 
ized in that said admission control system (18, 20) 
•is a distributed : system defined at least in part 
through interaction between said media push 
.engines (12).: / , : ■ 

3. The: media delivery system.of clainr 1 , charac?ter- 
-ized in.that said .media push engines (12) cqmmyni- 

r. ^cate. with said network (14) ...oyer different 
communication paths. . 

4. The media delivery. system of. claim 1, character- 
ized. in that said network. (14) is a connectionless 
network offering.best effort delivery services. 

5. The media delivery system of claim 1, character- 
ized in that said network (14) is the Internet. 

6. The media delivery system of claim 1, character- 
ized in that said media. client (16) and said media 
push engines (12) . participating in said multicast 

. group session employ Real-Time Transport Proto- 
col (RTP) for data transport. 

7. The media delivery system of claim 1, character- 
ized in that said media client (16) and said media 

•push engines (12) participating in said multicast 
group session employ Real-Time Control Protocol 
(RTCP) for session management. 



8. The media delivery system of claim 1, character- 
ized in that at least a portion of said substream 
components are replicated across several media 
push engines (12). 

9. The. media delivery system of claim 1, character- 
ized in that at least a portion of said substream 
components arar eplicated across first and second 
media push engines (12) and that said delivery sys- 
tem further, comprises congestion handling system 
for identifying one of said first and second media 
push engines (12) as the cause of congestion and 

-fot automatically invoking the other of said first and 
second^medi^. push .engines (12) to participate in 
said multicast group session. . 

10. The media delivery system of claim $, 2 character-, 
ized by data buffering system associated with said 

. media client (16) for storing ^t^tream components 
•.i. prior to reconstitutiqn> v ri r :0 . . x 

11-k The- medi^ctelivery system . of claim 1, ,character- 
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ized in that said admission control unit (18, 20) is 
further operative, in response to a request from said 
media client (16) to end a multicast group session, 
to instruct all media push engines (12) participating 
in said multicast group session to terminate the 5 
session. 

12. The media delivery system of claim 1, character- 
ized in that said admission control system (18, 20). 
includes an admission control unit (18) that main-S ; 10 ; 
tains a pooi of multicast session addresses for use/- -; 

in invoking multicast group sessions and wherein...' • 
said admission control unit (18) assigns a desig- 
nated multicast session address, selected from 
said pool, for use by said multicast group session:; 15 

13. The media delivery system of claim 12, character- 
ized in that said admission control unit (18) is fur- 
ther operative, in response to a request from said 
media client (16) to end a multicast group session, 20 
to return said designated multicast session address 

to said pool. 

14. The media delivery system of claim 12. character- 
ized in that said media client (16) and said media 25 
push engines (12) participate in said multicast 
group session exchange flow control messages 
without involving said admission control unit (18). 

15. The media delivery system of claim 1, character- 30 
ized in that between said media client (16) and 
every media push engine (12) participating in said 
multicast group session there is a unicast flow of 
datagrams containing real-time stream component 
data. 35 
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